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DETAILED ACTION 

Response to Amendment 

1. The amendment was received on 14 November 2005. 

2 . Claims 3 1 -7 1 are cancelled. 

3 . Changes to the Drawings (Figure 6) have been accepted. 

4. Because claim 60 has now been cancelled, the objection made to it in the previous action has 
been withdrawn. 

5. 35 U.S.C. 1 12 2 nd paragraph rejection made to claims 9, 10, 19 and 26 have been overcome and 
withdrawn. 

Information Disclosure Statement 

6. The information disclosure statements (IDS), submitted on 3 February 2006, 14 November 2005 
and 22 September 2005, have been considered by the examiner. 

Response to Arguments 

7. Applicant's arguments filed 14 November 2005 (page 14) have been fully considered but they are 
not persuasive. 

The only arguments with regard to reference de Cuetos (from pages 12-14 under Remarks) is 
when applicant argue on page 14 stating, "Although de Cuetos mentions a face detector and face tracker, 
there is no discussion or mention in de Cuetos of how the face detection is performed . . . Accordingly, for 
at least these reasons, Applicant submits that amended claim 1 is allowable over de Cuetos." 

The examiner respectfully disagrees. Claim 1 does not recite a method for detecting a face. 
There is nowhere in claim 1 a mention of how the face detection is performed. The claim 1 only required 
"automatically detecting a candidate area for a new face region in the frame." The arguments are 
unconvincing, as the limits in arguments are not recited in (at least) claim 1 . 
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Applicants' argue on page 18 with regard to reference of Wang stating "No where in Wang is 
there any discussion or mention that that Chi-squared values are used to verify that a candidate area 
includes a face. . . if the similarity value between a color histogram and an estimated color histogram is 
greater than a threshold value." The examiner respectfully disagrees. 

Wang discloses detecting scenes and summarizing video sequences. At col. 4 lines 9-35 Wang 
discloses generating a color histogram for each frame. The Wang discloses computing Chi-squared value 
with is compared to the predetermined threshold, verifying scene (candidate area) in the frame if the 
comparing value is either greater or exceeds the threshold value as disclosed at col. 4 lines 20-27. 

8. Applicant's arguments with respect to claims 1-8, 1 1-12, 13-18, 27-230 and 5-60 (on pages 1 1-21 
under Remarks) have been considered but are moot in view of the new ground(s) of rejection. Because of 
the amendment to claim 1 (changes the scope of the claim), the new grounds of rejection are submitted 
here within. 

Claim Rejections - 35 USC §102 

9. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis 
for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

10. Claims 1-2, 5, 13, 19-22, 24-27 and 29-30 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Dimitrova et al. (hereinafter, "Dimitrova") (US 6,754,389). 

With regard to claim 1 Dimitrova discloses a method (Figures 1-2) comprising: receiving a frame 
of content (10 in Figure 1); automatically detecting a candidate area in the frame that may include a face 
(300 in Figure 1, col. 3 line 9, lines 49-55); using one or more hierarchical verification levels to verify 
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whether a human face is in the candidate area (classification system 200, col. 3 lines 10-1 1, the 
classification system can be hierarchical classification system, col. 4 lines 33-35; col. 5 lines 54-58; col. 7 
lines 51-57. The classifier 200 also determines the classification depending on each segment (i.e., 
candidate area). For example, an anchor person, a weather newscaster, a commercial segment, etc., col. 3 
line 66 to col. 4 lines 1-52); indicating that the candidate area includes the face if the one or more 
hierarchical verification levels verify that a human face is in the candidate area (candidate area verifies 
whether it is a face or text according to the set of symbols, col. 6 lines 37-65); and using a plurality of 
cues to track each verified face in the content from frame to frame (col. 5 lines 6-9). 

With regard to claim 2 Dimitrova discloses video content as seen at elements 10 in Figure 1 and 
its respective portions in the specification. 

With regard to claim 5 Dimitrova discloses repeating automatic detecting in the even tracking of 
a verified face is lost at col. 9 lines 3-20. 

With regard to claim 13 Dimitrova discloses the candidate area including a face comprising: 
recording the candidate area in a tracking list (see Face Tracking System 300). 

Claim 19 is rejected the same as claim 1 . Thus, arguments similar to that presented above for 
claim 1 is equally applicable to claim 19. Claim 19 distinguishes from claim 1 only in that it recites 
predicting where a contour of the face will be; encoding a smoothness constraint that penalizes roughness; 
applying the smoothness constraint to a plurality of possible contour locations; and selecting the contour 
location having the smoothest contour as a location of the face in the frame. Dimitrova teaches these 
features at col. 8 lines 1-65. Please note the skin tone extractor and smoother 330, the predictive 
technique at lines 29-30 and selecting locations at lines 43-51. 

With regard to claims 20-21 Dimitrova discloses the smoothness constraint including both 
contour smoothness and region smoothness at col. 8 lines 8-10 and 17-23. 
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With regard to claim 22 Dimitrova discloses Hidden Markov Model (HMM) at col. 6 lines 37-45 
and at col. 7. 

With regard to claim 24 Dimitrova discloses adapting the predicting for the face in subsequent 
frames to account for changing color distributions (at col. 8 lines 29-30 - predictive techniques for 
determining each face trajectory, the trajectory including color at col 3 lines 61-65). 

With regard to claim 25 Dimitrova discloses adapting the predicting for the face in subsequent 
frames based on one or more cues observed in the frame (at col. 8 lines 29-30 - predictive techniques for 
determining each face trajectory, the trajectory including cues at col. 3 lines 53-65). 

Claim 26 is rejected the same as claim 19. Thus, arguments similar to that presented above for 
claim 19 is equally applicable to claim 26. 

With regard to claim 27 Dimitrova discloses tracking multiple possible locations at col. 8 lines 

30-55. 

With regard to claims 29-30 Dimitrova discloses using a particle filter to concurrently track the 
multiple possible locations at col. 8 lines 62-65. 

Claim Rejections - 35 USC §103 

1 1. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

12. Claim 3-4, 6-7, 1 1 and 14-18 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Dimitrova et al. (hereinafter, "Dimitrova") (US 6,754,389) in view of de Cuetos et al. (hereinafter, "de 
Cuetos") (US 6,754,373). 

With regard to claims 3 and 4 Dimitrova discloses a method as disclosed above in claim 1 and 
the arguments are not repeated herein, but are incorporated by reference. Dimitrova does not expressly 
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disclose the frame of content comprising a frame of audio content, de Cuetos discloses both audio and 
video content as seen at elements 18 and 26 in Figure 1 and at col.4 lines 1-20. At the time of the 
invention, it would have been obvious to a person of ordinary skill in the art to combine the teaching of 
Dimitrova with de Cuetos. The motivation for doing so is to have audio content rather than video as 
suggested by de Cuetos. Please note, in Dimitrova, the "other object tracking system" at element 500 in 
Figure 1 . Audio can be one of the other objects that can be tracked from the video stream 10. Therefore, 
it would have been obvious to combine Dimitrova with de Cuetos to obtain the invention as specified in 
claims 3-4. 

With regard to claim 6 de Cuetos discloses receiving the frame of content comprises receiving a 
frame of video content from a video capture device local to a system implementing the method (as seen in 
Figure 1, the system of local). 

With regard to claim 7 de Cuetos discloses receiving the frame of content comprises receiving 
the frame of content from a computer readable medium accessible to a system implementing the method 
(computer system disclosed in Figure 1). 

With regard to claim 11 it would have been obvious matter of design choice to modify the de 
Cuetos reference by having levels in the hierarchical verifications levels to include a level to verify a ROI 
faster but less accuracy than the other level since applicant has not discloses that having this coarse levels 
solves any stated problem or is for any particular purpose and it appears that the levels defied in claim 1 1 
would perform equally well with the levels defined as in claim 1 (as disclosed in de Cuetos). 

With regard to claim 14 de Cuetos discloses recording the candidate area in the tracking list 
comprising accessing a record corresponding to the candidate area and resetting a time since last 
verification of the candidate (see "Countdown timed out?", element 122 in Figure 2). 

With regard to claim 15 de Cuetos discloses the one or more hierarchical verification levels 
include a first level and a second level, and wherein using the one or more hierarchical verification levels 
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to verify whether the human face is in the candidate area comprises: checking whether, using the first 
level verification, the human face is verified as in the candidate area; and using the second level 
verification only if the checking indicates that the human face is not verified as in the candidate area by 
the first level verification (de Cuetos uses the classifier 32 to classify features at different level. Verifying 
continuously whether the person is talking or not and whether the person comes in front of a camera. See, 
col. 5 line 32 to col. 6 lines 1-30). 

Claim 16 recites similar features as claim 15. Thus, arguments similar to that presented above 
for claim 15 is equally applicable to claim 16. Note that de Cuetos discloses identifying human face from 
the ROI mentioned above. 

Claim 17 recites identical features as claim 1 1 . Thus, arguments similar to that presented above 
for claim 1 1 is equally applicable to claim 17. 

With regard to claim 18 Both de Cuetos and Dimitrova discloses plurality of cues as color 
(Dimitrova, col. 3 lines 64-65), edge, motion, and audio (de Cuetos, col. 3 lines 50 to col. 4 lines 1-7). 

13. Claim 12 is rejected under 35 U.S. C. 103(a) as being unpatentable over Dimitrova et al. 
(hereinafter, "Dimitrova") (US 6,754,389) in view of Wang et al. (hereinafter, "Wang") (US 5,805,733). 

With regard to claim 12 Dimitrova discloses the method as disclosed above in claim 1 and the 
arguments are not repeated herein, but are incorporated by reference. Dimitrova does not expressly 
disclose generating a color histogram of the candidate area; generating an estimated color histogram of 
the candidate area based on previous frames; determining a similarity value between the color histogram 
and the estimated color histogram; and verifying that the candidate area includes a face if the similarity 
value is greater than a threshold value. Wang discloses this at col. 4 lines 9-35. At the time of the 
invention, it would have been obvious to a person of ordinary skill in the art to combine the teaching of 
Wang with Dimitrova. The motivation for doing so is for computation efficiency, by reducing the search 
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space, because all scenes in a group of related scenes need not appear in the window at the same time as 
suggested by Wang at col. 2 lines 1-15. Therefore, it would have been obvious to combine Wang with 
Dimitrova to obtain the invention as specified in claim 12. 

14. Claim 23 is rejected under 35 U.S.C. 103(a) as being unpatentable over Dimitrova et al. 
(hereinafter, "Dimitrova") (US 6,754,389). 

With regards to claim 23, it would have been obvious matter of design choice to modify the 
Dimitrova reference by having Joint Probability Data Association Filter (JPDAF) since applicant has not 
discloses that having JPDAF solves any stated problem or is for any particular purpose and it appears that 
the Hidden Markov Model (HMM) would perform equally well with smoothness constraints. 

15. Claim 28 is rejected under 35 U.S.C. 103(a) as being unpatentable over Dimitrova et al. 
(hereinafter, "Dimitrova") (US 6,754,389) in view of Murakami et al. (hereinafter, "Murakami") (US 
6,798,834). 

With regard to claim 28 Dimitrova discloses the method as disclosed above in claim 1 and the 
arguments are not repeated herein, but are incorporated by reference. Dimitrova does not expressly 
disclose multiple hypothesis tracking technique to track the multiple locations. Murakami discloses 
multiple hypothesis tracking technique to track the multiple locations such as a moving person 201, 
motionless person 202, a desk 203, etc. At the time of the invention, it would have been obvious to a 
person of ordinary skill in the art to combine the teaching of Murakami with Dimitrova. The motivation 
for doing so is to provide a motion prediction circuit, which can correctly perform motion prediction and 
improve the efficiency of coding by coding an image as suggested by Murakami at col. 3 lines 20-3 1 . 
Therefore, it would have been obvious to combine Murakami with Dimitrova to obtain the invention as 
specified in claim 28. 
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16. Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over Dimitrova et al. 
(hereinafter, "Dimitrova") (US 6,754,389) in view of de Cuetos et al. (hereinafter, "de Cuetos") (US 
6,754,373) as applied to claims 1, 3-4, 6-7, 1 1 and 14-18 above, and further in view of Kellner (US 
6,539,099). 

With regard to claim 8 Dimitrova (modified by, "de Cuetos") discloses a method as disclosed 
above in claim 1 and the arguments are not repeated herein, but are incorporated by reference, de Cuetos 
discloses detecting whether there is motion in the frame and, if there is motion in the frame, then 
performing motion-based initialization to identify one or more candidate areas (col. 3 lines 50-67); 
detecting whether there is audio in the frame, and if there is audio in the frame, then performing audio- 
based initialization to identify one or more candidate areas (col 4 lines 1-50). Neither Dimitrova nor de 
Cuetos expressly disclose using, if there is neither motion nor audio in the frame, a fast face detector to 
identify one or more candidate areas. Kellner discloses face detector at col. 4 lines 45-52. At the time of 
the invention, it would have been obvious to a person of ordinary skill in the art to combine the teaching 
of Kellner with Dimitrova and de Cuetos. The motivation for doing so is to allow more individuals to 
participate in the same video session as suggested by Kellner at col. 3 lines 1-12. Therefore, it would have 
been obvious to combine Kellner with Dimitrova and de Cuetos to obtain the invention as specified in 
claim 8. 

Allowable Subject Matter 

17. Claims 9-10 are allowed. 

The closest prior art to Dimitrova, de Cuetos, Wang, Kellner, and Murakami are directed to the 
method as disclosed in an independent claim 1. However, the closest prior art fails to disclose anything 
about determining whether there is motion at a plurality of pixels on a plurality of lines across the frame; 
generating a sum of frame differences for each possible segment of each of the plurality of lines; 
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selecting, for each of the plurality of lines, the segment having the largest sum; identifying a smoothest 
region of the selected segments; checking whether the smoothest region resembles a human upper body; 
and extracting, as the candidate area, a portion of the smoothest region that resembles a human head as 
disclosed in claim 9. It is for these reasons in combination with all the other elements of the claim that 
claim 9 would be allowable if rewritten in independent form including all of the limitation of the base 
claim and any intervening claims. Claim 10 is allowable for the same reason as claim 9. 

Conclusion 

18. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office 
action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is 
reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE MONTHS from 
the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing 
date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH 
shortened statutory period, then the shortened statutory period will expire on the date the advisory action 
is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later than SIX 
MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the examiner should 
be directed to Shefali D. Patel whose telephone number is 571-272-7396. The examiner can normally be 
reached on M-F 8:00am - 5:00pm (First Friday Off). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Jingge Wu can be reached on (571) 272-7429. The fax phone number for the organization where this 
application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent Application 
Information Retrieval (PAIR) system. Status information for published applications may be obtained 
from either Private PAIR or Public PAIR. Status information for unpublished applications is available 
through Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 

Shefali D Patel 
Examiner 
Art Unit 2621 
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